67 research outputs found
Learning Rank Reduced Interpolation with Principal Component Analysis
In computer vision most iterative optimization algorithms, both sparse and
dense, rely on a coarse and reliable dense initialization to bootstrap their
optimization procedure. For example, dense optical flow algorithms profit
massively in speed and robustness if they are initialized well in the basin of
convergence of the used loss function. The same holds true for methods as
sparse feature tracking when initial flow or depth information for new features
at arbitrary positions is needed. This makes it extremely important to have
techniques at hand that allow to obtain from only very few available
measurements a dense but still approximative sketch of a desired 2D structure
(e.g. depth maps, optical flow, disparity maps, etc.). The 2D map is regarded
as sample from a 2D random process. The method presented here exploits the
complete information given by the principal component analysis (PCA) of that
process, the principal basis and its prior distribution. The method is able to
determine a dense reconstruction from sparse measurement. When facing
situations with only very sparse measurements, typically the number of
principal components is further reduced which results in a loss of
expressiveness of the basis. We overcome this problem and inject prior
knowledge in a maximum a posterior (MAP) approach. We test our approach on the
KITTI and the virtual KITTI datasets and focus on the interpolation of depth
maps for driving scenes. The evaluation of the results show good agreement to
the ground truth and are clearly better than results of interpolation by the
nearest neighbor method which disregards statistical information.Comment: Accepted at Intelligent Vehicles Symposium (IV), Los Angeles, USA,
June 201
Image Inpainting with Learnable Feature Imputation
A regular convolution layer applying a filter in the same way over known and
unknown areas causes visual artifacts in the inpainted image. Several studies
address this issue with feature re-normalization on the output of the
convolution. However, these models use a significant amount of learnable
parameters for feature re-normalization, or assume a binary representation of
the certainty of an output. We propose (layer-wise) feature imputation of the
missing input values to a convolution. In contrast to learned feature
re-normalization, our method is efficient and introduces a minimal number of
parameters. Furthermore, we propose a revised gradient penalty for image
inpainting, and a novel GAN architecture trained exclusively on adversarial
loss. Our quantitative evaluation on the FDF dataset reflects that our revised
gradient penalty and alternative convolution improves generated image quality
significantly. We present comparisons on CelebA-HQ and Places2 to current
state-of-the-art to validate our model
RGB-D Mapping and Tracking in a Plenoxel Radiance Field
Building on the success of Neural Radiance Fields (NeRFs), recent years have
seen significant advances in the domain of novel view synthesis. These models
capture the scene's volumetric radiance field, creating highly convincing dense
photorealistic models through the use of simple, differentiable rendering
equations. Despite their popularity, these algorithms suffer from severe
ambiguities in visual data inherent to the RGB sensor, which means that
although images generated with view synthesis can visually appear very
believable, the underlying 3D model will often be wrong. This considerably
limits the usefulness of these models in practical applications like Robotics
and Extended Reality (XR), where an accurate dense 3D reconstruction otherwise
would be of significant value. In this technical report, we present the vital
differences between view synthesis models and 3D reconstruction models. We also
comment on why a depth sensor is essential for modeling accurate geometry in
general outward-facing scenes using the current paradigm of novel view
synthesis methods. Focusing on the structure-from-motion task, we practically
demonstrate this need by extending the Plenoxel radiance field model:
Presenting an analytical differential approach for dense mapping and tracking
with radiance fields based on RGB-D data without a neural network. Our method
achieves state-of-the-art results in both the mapping and tracking tasks while
also being faster than competing neural network-based approaches.Comment: *The two authors contributed equally to this pape
- …